Bounds on the Capacity of Deletion

I. Summary In this paper, we develop bounds on the achievable rate for deletion channels. Deletion channels occur when symbols are randomly dropped, and a subsequence of the transmitted symbols is received. Our initial motivation for studying these channels arose in the context of information transmission over nite-buuer queues in packet-switched networks, where the receiver does not have access to side-information on which packets were dropped. This also motivated our study on the eeect of large alphabet sizes on the achievable rate. In deletion channels, unlike erasure channels, there is no side-information about which subsequence is received. Clearly, the capacity of the erasure channel is therefore a simple upper bound for the capacity of the deletion channel. The natural question which arises is about how much worse (in terms of information transmission rate) a deletion channel is compared to an erasure channel. This might shed light on the amount of \redundancy" actually needed for transmission over packet loss channels. The deletion channel is a special case of inser-tion/deletion/substitution channels 1 which model the eeect of synchronization errors and have a long history ((1, 2] and references therein.) Even in the presence of memoryless deletions , there is no single-letter characterization for achievable rates. All the published literature deals with the binary alphabet case. For memoryless deletion channels 1, 2] showed that C del 1 H0(p d); p d 0:5 (1) where H0(p d) = (1p d) log(1p d)p d log(p d) is the binary entropy function. We provide an alternative proof for this result which also yields lower bounds for larger (non-binary) alphabet sizes and when the deletion process is stationary and ergodic. We also derive bounds that improve (1) by using codebooks with memory. Our main result is that the achievable rate in deletion channels diiers from that of erasure channels by at most H0(p d) p d log K K1 bits, where p d is the deletion probability, K is the alphabet size and H0() is the binary entropy function. We sharpen these bounds by giving a characterization of achievable rates using input codebooks with memory for the non-binary deletion channel. These lower bounds, when specialized for the binary deletion channel, improve the bounds reported in (1). II. Problem Statement and Main Result The K-ary deletion channel is deened as follows. Di = 1 indicates that the i-th symbol of x is deleted, and Di = 0 indicates …